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DETAILED ACTION 



1. This Office action for US Patent Application 10/802,428 is in response to 
communications filed 22 February 2008, in reply to the Notice of a Non-Compliant 
Amendment of 04 February 2008. Currently, claims 1-3, 5-15, 17-25, 27, and 29 are 
pending. Claims 4, 16, 26, 28, and 30 have been canceled. Applicant notes that the 
limitations of the canceled claims have been incorporated into the independent claims. 

2. In the previous Office action, claims 1-12 were rejected under 35 U.S.C. 101 as 
non-statutory. Claims 1-7, 10-12, 13-19, and 22-26 were rejected under 35 U.S.C. 
103(a) as obvious over "Motion-based Segmentation Using a Thresholded Merging 
Strategy on Watershed Segments (de Smet et al.) in view of "K-Harmonic Means-A 
Data Clustering Algorithm" (Zhang et al.). Claims 8, 9, 20, and 21 were rejected under 
35 U.S.C. 103(a) as obvious over de Smet et al. in view of Zhang et al., and in further 
view of "A Video Segmentation Algorithm for Hierarchical Object Representations and 
its Implementation" (Herrmann et al.). Claims 27-30 were rejected under 35 U.S.C. 
103(a) as obvious over de Smet et al. in view of Zhang et al., and in view of US Patent 
6,084,912 A (Reitmer et al.). The drawings were objected to as having reference 
numbers inconsistent with the specification. A Requirement for Information was made 
under the provisions of 37 CFR 1 .105. 
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Requirement for Information - 37 CFR 1.105 

3. Applicant's supporting documents filed in response to the Requirement for 
Information under 37 CFR 1 .105 have been considered. 

Response to Amendment 

4. Applicant's amendment to the specification has been fully considered. The 
objection to the drawings is withdrawn. 

5. Applicant's amendment to claim 1 has been fully considered but is insufficient to 
overcome the rejection under 35 U.S.C. 101 due to language in the specification. 

Response to Arguments 

6. Applicant's arguments filed 15 November 2007 have been fully considered but 
they are not persuasive. Applicant argues that the combination of de Smet et al. and 
Zhang et al. does not include the claimed limitations of "initializing regression functions 
for each of the K clusters to estimate the centers of motion for the data points", 
"calculating the distances from each data point to each of the K regression functions", 
and "calculating a membership probability and a weighing factor for each data point 
based on distances between the K regression functions and each data point", and that 
because of this alleged gap in the combination, there is no motivation to combine de 
Smet et al. and Zhang et al. 

Regarding "initializing regression functions", it is respectfully submitted that while 
Zhang et al. states that an advantage of KHM clustering is an insensitivity to 
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initialization of the centers, KHM still "starts with an initialization of the center positions" 
(page 2). In the first iteration of the KHM algorithm, when a set of initial positions of 
centers is initialized, the distances d from data points x and centers m are calculated, 
"and then the new positions of the centers are calculated" (page 5). This initial new 
determination of center positions in the first iteration of the KHM function is an 
"estimate" of the centers of motion for the data points according to the present 
invention. 

Regarding "calculating the distances", Applicant states that "the invention uses 
multiple K regression functions", while "Zhang uses a single performance function" 
(Arguments, page 13). However, the "each of the K regression functions" in the 
previous Office action was interpreted as the specific instance of KHM for each center 
point, since the present invention is directed to "using a K-Harmonic Means function", 
which was read as in the singular. Therefore, the calculation of a distance d between 
each data point and every center point is the claimed distance calculation according to 
the present invention. 

Regarding "calculating a membership probability", it is respectfully submitted that 
the calculation of q(i,k) in Zhang et al. is equivalent to probability function p(Z,z) in the 
specification, and regarding calculating "a weighing factor", the calculation of p(i,k) in 
Zhang et al. is equivalent to weighting function a(z) in the specification, albeit with 
differences in notation. 
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Therefore, in view of the above, it is respectfully submitted that the combination 
of de Smet et al. and Zhang et al. discloses every claimed limitation of the present 
invention. 



Claim Rejections - 35 USC § 101 



7. 35 U.S.C. 101 reads as follows: 



Whoever invents or discovers any new and useful process, machine, manufacture, or composition of 
matter, or any new and useful improvement thereof, may obtain a patent therefor, subject to the 
conditions and requirements of this title. 



8. Claims 1-12 are rejected under 35 U.S.C. 101 because the claimed invention is 



directed to non-statutory subject matter. 



The relevant portions of the USPTO "Interim Guidelines for Examination of 
Patent Applications for Patent Subject Matter Eligibility (Official Gazette Notice of 22 
November 2005), Annex IV, read as follows: 

In contrast, a claimed computer-readable medium encoded with a computer program is a 
computer element which defines structural and functional interrelationships between the 
computer program and the rest of the computer which permit the computer program's 
functionality to be realized, and is thus statutory. See Lowry, 32 F.3d at 2583-84, 32 USPQ2d 
at 1035. 



Claims that recited nothing but the physical characteristics of a form of energy, such as a 
frequency, voltage, or the strength of a magnetic field, define energy or magnetism, per se, 
and as such are nonstatutory natural phenomena. See O'Reilly, 56 U.S. (15 How.) at 112- 
114. Moreover, it does not appear that a claim reciting a signal encoded with functional 
descriptive material falls within any of the categories of patentable subject matter set forth in 
Sec. 101. 



...a signal does not fall within one of the four statutory classes of Sec. 101. 



...signal claims are ineligible for patent protection because they do not fall within any of the 
four statutory classes of Sec. 101 . 



Claims 1-12 are drawn to a "computer-readable medium" storing functional 
descriptive material. Normally, the claims would be statutory. However, the 
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specification, at paragraph 0088, defines the claimed computer-readable medium as 
encompassing statutory material such as a "data storage", as well as non-statutory 
subject matter such as a "carrier" or "communications devices". 

A signal embodying functional descriptive material is neither a process nor a 
product (i.e., a tangible "thing") and therefore does not fall within one of the statutory 
classes of §101. Rather, a "signal" is a form of energy, in the absence of any physical 
structure or tangible material. See In re Nuijten, 84 USPQ2d 1495 (Fed. Cir. 2007, en 
banc denied 2008). 

Because the full scope of the claims as properly read in light of the disclosure 
encompasses non-statutory subject matter, the claims as a whole are non-statutory. 

Claim Rejections - 35 USC § 103 

9. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 
obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 

10. Claims 1-3, 5-7, 10-15, 17-19, and 22-25 are rejected under 35 U.S.C. 103(a) as 
being unpatentable over "Motion-based Segmentation Using a Thresholded Merging 
Strategy on Watershed Segments (de Smet et al.) in view of "K-Harmonic Means-A 
Data Clustering Algorithm" (Zhang et al.). De Smet et al. discloses using an iterative 
segment-merging technique to determine motion information for an image (abstract). 
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Regarding claims 1, 13, and 25, in de Smet et al., an initial motion field is first 
determined with a block-matching technique on 4 x 4 blocks (§ 2.1). These initial block 
motion vectors are used for the initial segmentation (§ 2.3). Then, the step of 
performing the block-based motion estimation for use in the first segmentation 
corresponds with the step of "providing data points". Next, the segments are iteratively 
merged according to similar or shared motion, according to the K-means clustering 
algorithm (§ 2.3). This corresponds with the step of "clustering the data points". When 
this process is finished, the result is a series of large segments corresponding to distinct 
moving regions of an image, each with an associated motion vector (§ 2.3). This 
corresponds with the step of "providing motion estimation". However, the present 
invention specifies performing regression clustering according to a K-Harmonic Means 
function, which is not the same as the K-means function of de Smet et al. 

Zhang et al. discloses the K-Harmonic Means data clustering algorithm. 
Regarding claim 1 , 3, and 25, Zhang et al. teaches selecting K centers m(l) from N data 
points x(i) (pg. 1), initializing center points (pg. 2) and performing an initial iteration (pg. 
5), calculating distance d(i,l) between data point x(i) and center point m(l) (pg. 4), 
calculating membership probability q(i,k) and weighting function p(i,k) (pg. 5), 
recursively calculating new m(k), (pg. 5), and stopping when the recursively-calculated 
performance value stabilizes, that is, when its change with each iteration becomes small 
(P9- 5). 

De Smet et al. discloses the claimed invention except for performing a K- 
Harmonic Means function to perform regression clustering. Zhang et al. teaches that it 
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was known to perform data clustering with the K-Harmonic Means function. Therefore, 
it would have been obvious to one having ordinary skill in the art at the time the 
invention was made to perform clustering based on a K-Harmonic means function, as 
taught by Zhang et al., rather than a linear function such as the K-means function of De 
Smet et al., since Zhang et al. states in the abstract that K-harmonic clustering is less 
sensitive to detrimental effects from sub-optimal initialization that conventional 
clustering techniques. 

Regarding claims 2, 3, 14, and 15, as previously mentioned, de Smet et al. 
produces a motion vector for each segment in an image (§ 2.3). As a result, the most 
important moving areas are determined (§ 3). Regarding claims 10 and 22, in de Smet 
et al., pixels are set as (x,y,t) triples, with x and y as spatial coordinates and t as a time 
coordinate (§ 2.2). 

Regarding claims 11 and 23, de Smet et al. illustrates motion fields (figures 3-6). 
Although these motion fields are not shown as overlaid on the images, the examiner 
takes Official Notice that it was well-known in the art at the time of the invention to 
display a motion field superimposed on an image to provide a visual representation of 
motion vectors. 

Regarding claims 12 and 24, de Smet et al. illustrates highlighted motion 
segments overlaid on an image (figures 11 and 12). 

Regarding claims 5 and 17, in Zhang et al., a clustering in which initialization is 
randomized is described (pg. 11). 
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Regarding claims 6 and 18, in Zhang et al., KHM is described as a recursive 
function that performs until stabilization is achieved (pg. 5). 

Regarding claims 7 and 19, insensitivity to initialization is an inherent result of the 
K-Harmonic means algorithm (abstract). 

11. Claims 8, 9, 20, and 21 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over de Smet et al. in view of Zhang et al. as applied to claims 1 and 13 
above, and further in view of "A Video Segmentation Algorithm for Hierarchical Object 
Representations and its Implementation" (Herrmann et al.). Claims 8, 9, 20, and 21 
disclose using color information to segment images, but de Smet et al. only discloses 
"standard watershed techniques" (§ 2.2) to perform the initial segmentation without 
providing details. 

Herrmann et al. discloses a method for image segmentation to extract objects 
from a moving image. Regarding claims 8, 9, 20, and 21, after an initial block-matching 
motion estimation, similar to de Smet et al., images are segmented according to specific 
color information, followed by shape analysis, and lastly motion analysis to merge 
regions to determine objects (§ II). This corresponds with the claimed "predetermined 
criteria". In color analysis, a region is determined as homogeneous if the pixel 
difference in the region is below a threshold. Homogenous, connected areas are 
determined as "quasi-flat zones". These quasi-flat zones are further processed and 
become the basis for further segmentation (§ II. B). This corresponds with "portioning 
data according to color". 
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De Smet et al., in combination with Zhang et al., disclose a majority of the 
features of claims 8, 9, 20, and 21 as discussed above, the claimed invention except for 
color segmentation. Herrmann et al. teaches that it was known to segment a moving 
image according to color. Therefore, it would have been obvious to one having ordinary 
skill in the art at the time the invention was made to determine image segmentation by 
color as taught by Herrmann et al., since Herrmann et al. teaches in page 205, third 
paragraph, that color analysis produces the most accurate type of segmentation. 

12. Claims 27 and 29 are rejected under 35 U.S.C. 103(a) as being unpatentable 
over de Smet et al. in view of Zhang et al., and in view of US Patent 6,084,912 A 
(Reitmeier et al.) Claims 27 and 29 are in means-plus-function format, complying with 
35 U.S.C. 112, sixth paragraph. Accordingly, the "system" of claims 27 and 29 will be 
limited to a general-purpose computer such as a PC, as illustrated in figure 5 of the 
specification of the present invention. Although it is implied that the algorithms of de 
Smet et al. and Zhang et al. are computer-operated, neither de Smet et al. nor Zhang et 
al. explicitly teach this. 

Reitmeier et al. discloses a video encoder. This encoder may operate on 
MPEG-4 video (column 1: line 57), as specified in paragraphs [0006] and [0007] of the 
present invention as a codec on which the present invention is applied. Regarding 
claim 27, the encoder of Reitmeier et al. may operate as a software application on a 
general-purpose computer (column 2: lines 64-67). Regarding claim 29, Reitmeier et al. 
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discloses frame memory 155, which stores decoded reference frames for motion 
compensation (column 7: line 27), as is well-known in MPEG encoders. 

De Smet et al., combined with Zhang et al., discloses the claimed invention 
except for encoding video on a general-purpose computer. Reitmeier et al. teaches that 
it was known to implement an MPEG-4 encoder as software embedded on a computer. 
Therefore, it would have been obvious to one having ordinary skill in the art at the time 
the invention was made to encode the system of de Smet et al. and Zhang et al. in a 
software MPEG encoder embedded on a computer, as taught by Reitmeier et al., in 
order to perform computationally complex functions such as motion compensation, 
quantization, and variable-length encoding inherent in the video coding process. 

Conclusion 

13. The prior art made of record and not relied upon is considered pertinent to 
applicant's disclosure. "Trajectory Clustering with Mixtures of Regression Models" 
describes performing EM clustering in various applications, including video (pp. 12-16). 

14. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of time 
policy as set forth in 37 CFR 1 .136(a). 

A shortened statutory period for reply to this final action is set to expire THREE 
MONTHS from the mailing date of this action. In the event a first reply is filed within 
TWO MONTHS of the mailing date of this final action and the advisory action is not 
mailed until after the end of the THREE-MONTH shortened statutory period, then the 
shortened statutory period will expire on the date the advisory action is mailed, and any 
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extension fee pursuant to 37 CFR 1.136(a) will be calculated from the mailing date of 
the advisory action. In no event, however, will the statutory period for reply expire later 
than SIX MONTHS from the mailing date of this final action. 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to David N. Werner whose telephone number is (571)272- 
9662. The examiner can normally be reached on MWF from 9:00-6:30, TR from 9:00- 
4:30. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Mehrdad Dastouri, can be reached on (571) 272-7418. The fax phone 
number for the organization where this application or proceeding is assigned is 571- 
273-8300. 

Information regarding the status of an application may be obtained from the 
Patent Application Information Retrieval (PAIR) system. Status information for 
published applications may be obtained from either Private PAIR or Public PAIR. 
Status information for unpublished applications is available through Private PAIR only. 
For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic 
Business Center (EBC) at 866-217-9197 (toll-free). If you would like assistance from a 
USPTO Customer Service Representative or access to the automated information 
system, call 800-786-9199 (IN USA OR CANADA) or 571-272-1000. 
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Supervisory Patent Examiner, Art Unit 2621 



